-
Notifications
You must be signed in to change notification settings - Fork 53
Nextclade dataset reference trees #1249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
12b5505 to
b9c711c
Compare
|
Two linting errors in CI did not appear locally for me… ah, because I ran In any case, the linting didn't like two places I use I used the I guess I'll tell eslint to go stuff it? |
b9c711c to
e1d3693
Compare
|
Punting on the logo issue as part of #870 (comment) instead. |
|
A good conversation about forwards compatibility with Nextclade's index and breaking changes is happening in Slack, started by @ivan-aksamentov. |
|
Planning to pick this up after #1274 is merged. |
Removes the mixed use of a versionDescriptor function param and instance property in favor of just the instance property. Moves Resource.versionDescriptor()'s JSDoc where it's supposed to be: just prior to the method instead of just inside the method.
…subresource() method This makes .subresources() work as designed (though no behaviour change is expected). This change was missed in "sources: Refactor subresource construction into Resource base class" (ef13cd6), which introduced the static .Subresource property.
This reverts commit d467229. I'm in this canonicalization code again for a new source and want to address the query param handling centrally (rather than proliferate it further) as mentioned in my original review.¹ Doing so includes going back to a cleaner pathBuilder interface for routes that don't need "req". ¹ <#793 (review)>
e1d3693 to
4bb7fd0
Compare
Preserving by default makes more sense for most of our routes, and for routes where it does not (e.g. /charon/…), adjusting the query via the callback is now simpler and more declarative like the other routes. Motivated by being in this canonicalization code again for a new source and wanting to address the query param handling centrally (rather than proliferate it further) as mentioned in my original review.¹ No external behaviour change; effectively a different approach to "Preserve URL queries across redirects" (1c7838a). ¹ <#793 (review)>
The same async wrapper is applied to the Express application instance, Router instances, and Route instances (i.e. from .route()). The latter don't have .use(). Breaks much earlier at runtime (e.g. when setting up routes), crashing the server, rather than allowing the server to start and erroring on requests to the affected route.
We use .all() to avoid repeating middleware for each HTTP method on a route. I need to start using async middleware in those places.
Useful to be flexible here so the callback can be written less awkwardly, and necessary to fix a bug (coming soon).
…rent Source
Fixes a subtle bug where changing the dataset *in Auspice* to a
non-canonical path caused a redirection that switched sources, e.g.
staging → core.
For example, if you were on /staging/measles/genome and switched measles
to enterovirus, Auspice made a /charon/getDataset request for
/staging/enterovirus which *should* have canonicalized to
/staging/enterovirus/d68/genome but *was* canonicalized to
/enterovirus/d68/genome. This was demonstrable by:
$ curl -IL https://nextstrain.org/charon/getDataset?prefix=/staging/enterovirus -so /dev/null -w '%{url_effective}\n'
https://nextstrain.org/charon/getDataset?prefix=enterovirus%2Fd68%2Fgenome
Note that the "prefix" query param was missing "staging/".
So we can more easily do external lookups for canonicalization.
So we can more easily construct or lookup the value based on external information. I didn't bubble this up to Resource.baseName because it was not necessary for my purposes, but I wouldn't be surprised if we find a need to do that in the future.
Missed in "Derive inventory path from key in manifest" (4b2257c).
I will use this in new code that's forthcoming.
…not name This bug had no visible effect on "core" datasets where the URL path (e.g. /a/b/c) and name (a/b/c) are equivalent. For other sources, however, like "staging", the URL path (e.g. /staging/a/b/c) and name (a/b/c) differ. The bug remained latent because historical versions were only ever enabled for the "core" source. I noticed when adding UI for a "nextclade" source that also enabled versions.
…to sort The "sortingName" was computed for every resource by _sortableName(), but then not used in the actual sort operations. Datasets with recency timeframes (e.g. 2y, 6y, 12y, etc) are now sorted as expected in the UI instead of being ASCII-betical.
…) construction Parameterize it so that callers can lump resources in the UI by names other than the first slash-delimited part of the resource name. I'll be using this shortly.
…pName" I'll be using this shortly to statically sort one set of resource groups after another.
4bb7fd0 to
e213c5e
Compare
e213c5e to
a679094
Compare
The new Nextclade source class and related classes in the Source/Resource/Subresource framework provide access the "latest" Nextclade dataset reference trees and resolve a myriad of supported aliases. The resource indexer is extended with a new "resource collection" for Nextclade that essentially transforms the existing Nextclade indexes into what our resource indexer expects. This, in turns, fits into existing resourceIndexer/ and src/resourceIndex.js code to provide access to historical versions of Nextclade dataset reference trees. Bumps the resource index version to v9 since changes were made to resourceIndex/. The static-site/app/nextclade/… files were largely copied from static-site/app/staging/… and then modified to refer to the "nextclade" source instead. There is a lot of boilerplate and duplication. But that appears to be the way it's been done for other usages, and I don't have time to make it better so close to the eve of my departure. The biggest differences are in the resources.tsx file. Resolves: <#1156>
a679094 to
4e59d79
Compare
|
Changes have been rebased onto latest Re: e1d3693
I've created ##1281 to address this separately. If it looks like a good direction to others, I'll bring those changes into this PR via rebase. |
+1 for this direction! |


See commit messages for details.
Some previews to check out:
nextstrain remote ls https://nextstrain-s-trs-nextcl-e9eixu.herokuapp.com/nextcladenextstrain remote download https://nextstrain-s-trs-nextcl-e9eixu.herokuapp.com/nextclade/nextclade/mpox/clade-iibResolves: #1156
Checklist
Omit Nextstrain logo for community datasets (although our listings for Groups do not do this yet either)see Port new resource-listing “cards” UI for use to display datasets on /groups #870 (comment)